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Abstract 

Literature consistently documents a positive, direct effect of students’ attitudes on learning 
(Lizzio, Wilson, & Simons, 2002). Hence, accounting studies describing active learning 
activities often report student attitudes as evidence of efficacy (e.g., Matherly & Burney, 
2013), but rely on single-item instead of multi-item scales. This practice in accounting 
impedes evaluation of active learning activities or testing of empirical models. Thus, we (1) 
develop scales capturing students’ attitudes, (2) use qualitative inquiry to validate our 
scales, and (3) empirically test our scales using Biggs’ (1989) 3P Model. These scales will 
enable this stream to mature through more consistent constructs and sophisticated 
modeling. 

Keywords: authentic learning, active learning, students’ attitudes, scale development, 
assessment, explanatory mixed methodology 
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Education literature within an accounting context continues to rely on single item 
questions in evaluating pedagogical innovations. However, this practice is contrary to good 
principles of survey methodology. For example, Fuchs and Diamantopoulos (2009) 
designate criteria for situations where single-item versus scale measures are appropriate. 
Specifically, they contend that measures intended to capture attitudinal responses mandate 
the use of multi-item scales. Drawing from their research, we assert that the literature 
investigating the efficacy of active learning would benefit from the development of scales to 
capture student perceptions and attitudes. 

One area of accounting education research immersed in the reliance on single items 
is active, authentic learning. Actively engaging students in their learning “is increasingly 
recognized as a vital ingredient in the university context” (Hawtrey, 2007, p. 143). Prince 
(2004) identifies three broad categories of benefits related to active learning: increased 
content knowledge (i.e., cognitive domain), enhanced students’ attitudes (i.e., affective 
domain), and improved results regarding “pragmatic items [such] as student retention in 
academic programs” (p. 224). This trend toward more active learning has impacted 
accounting academics at the university level through various calls to shift from a passive 
teaching approach to one that encourages students’ active participation in the learning 
process (Fowler, 2006). 

Prior education research demonstrates that learning environments have direct effects 
on students’ content mastery (Fizzio, Wilson, & Simons, 2002). Thus, the accounting 
education literature frequently relies on positive student attitudes as a desirable outcome 
associated with active learning activities (Apostolou et ah, 2013). For example, in 2011, 
our review showed that 63% of the 41 active learning articles published in the four leading 
accounting education journals reported students’ attitude toward (perception of) an activity 
as evidence of the activity’s efficacy. The questions asked on the evaluations generally fell 
into two categories: general questions about the active learning experience and specific 
questions about the activity’s learning objectives. In these studies, attitudinal student 
responses are captured and evaluated using single-item measures. 

In this article, we build on prior literature to develop four scales to measure students’ 
attitudes about the general active learning experience. Our hope is that use of these scales 
by accounting education researchers will increase the rigor of research in this stream. It is 
in this spirit that we have added a qualitative component to our determination of validity 
and reliability. 
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LITERATURE REVIEW 

Principles of Survey Methodology 

This paper focuses on the construct measurement of single versus multiple survey 
items. Pedhazur and Schmelkin (1991), when discussing construct validation, state, “The 
use of a single indicator for the measurement of a construct...almost always poses 
insurmountable problems, because it is not possible to identify and separate the different 
sources of variability of the indicator in question” (p. 56). While this statement justifies 
the reliance on multiple items for construct measurement, the topic was revisited by 
Bergkvist and Rossiter (2007), who contend that single-items are appropriate in many 
instances. They examined the application of both single and multiple item scales and 
documented that predictive validity is not compromised by relying on single-item measures 
for attributes that are “concrete and singular.” Bergkvist and Rossiter summarize the 
arguments from the extant literature for using multiple items. The item in their list most 
relevant to this study is that multiple items are “necessary if [the] object is abstract or [the] 
attribute is abstract” (p.l78). Ultimately, Bergkvist and Rossiter assert that their study 
supports the use of single-item measures for constructs such as attitude, which they consider 
“doubly concrete.” In their terms, “doubly concrete” occurs with a simple object (i.e., an 
advertisement) and simple attribute (i.e., “liking the advertisement”). 

In contrast, Diamantopoulos (2005, p. 2) contends that “this line of 
argument...goes against the fact that constructs, by their very nature, are abstract entities.” 
Fuchs and Diamantopoulos (2009) specify eight criteria to determine when single items 
can yield reliable results. When attempting to capture an abstract construct, Fuchs and 
Diamantopoulos repeat a general guideline that “the use of multiple-item measures is 
required, because most constructs, by definition, are too complex to be measured effectively 
with a single item” (p. 202). Furthermore, single-item measures may be too vague for 
respondents to incorporate all facets of the construct into their evaluation. 
Diamantopoulos, Sarstedt, Fuchs, Wilczynski, and Kaiser (2012) summarize their review 
of the marketing literature as demonstrating “that the predictive validity of single items 
varies considerably across different [concrete] constructs and stimuli objects” (p. 434). 
Ultimately, the results of their simulations suggest that in regards to predictive validity, 
multi-item scales are clearly superior to single items. 

Active, Authentic Learning Experiences 

Active learning is a component of the authentic learning educational movement that 
seeks to increase students’ motivation and learning (Ma & Lee, 2012, p. 272). The 
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structural ideas of authentic learning in education are often credited to Piaget and other 
Constructivists (Schreiber & Valle, 2013) who believed that learning is an active, not 
passive, process, which connects new knowledge to existing knowledge through interaction 
and analyses, within a context that will be applicable in a work setting (Barr & Tagg, 1995; 
Brown, Collins, & Duguid, 1989; Lave, 1988; Piaget, 1954, 1974). Prince (2004) further 

defines active learning “as any instructional method that engages students in the learning 
process...While this definition could include traditional activities such as homework, in 
practice active learning refers to activities that are introduced into the classroom’ ’ (p. 223). 
With active learning, students are involved in the learning process through an activity that 
requires them to “think about what they are doing” (Smith & Cardaciotto, 2011, p. 57). 
Thus, active learning goes beyond simply participating in an activity in that students must 
engage in deeper intellectual thought, such as evaluation, synthesis, and reflection (Bonwell 

& Eison, 1991; Fink, 2003; Smith & Cardaciotto, 2011). 

Rule (2006) evaluated 45 articles in the fields of education, as well as arts and 
sciences, to establish parameters to determine authentic learning. The four focus areas for 
an authentic learning experience are: 

1. engages students in problems that simulate the “work of professionals,” 

2. employs critical thinking skills using open-ended inquiry, 

3. involves a “community of learners,” and 

4. incorporates activities that are learner-centered and commonly self-directed. 

Authentic learning positions students to apply the concepts throughout the learning 
process (Ma & Lee, 2012). Research in the fields of finance and accounting report increased 
undergraduate student satisfaction and experiences of deep learning resulting from authentic 
learning activities (Brimble, Cameron, Freudenberg, Fraser, & MacDonald, 2012; Hui & 
Koplin, 2011; Killian, Huber, & Brandon, 2012; Turner & Baskerville, 2011). 

Relevance of Student Attitudes and Perceptions 

Learning environment research receives considerable attention in the broader 
education literature (Fraser, 1998; Opdenakker & Minnaert, 2011; Senocak, 2009; Walker 
& Fraser, 2005). Lizzio et al. (2002) rely on Biggs’ (1989) 3P Model, which describes 
the learning process as an interaction among presage (learning environment and student 
characteristics), process (students’ learning style), and product (learning outcomes). 
Within this model, learning environment encompasses situational characteristics, such as 
teaching method and course structure. Lizzio et al. (2002) indicate a general proposition 
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that it is students’ perceptions toward “their learning environment, in light of their 
motivations and expectations, which determine how situational factors influence approaches 
to learning and learning outcomes” (p. 28). 

SURVEY DEVELOPMENT 

Selection of Articles 

To construct multi-item scales measuring student perceptions of active learning 
within an accounting setting, we developed a survey that students completed at the end of 
the semester, which included four active learning activities. We wanted to include a broad 
set of items that accounting educators have recognized as important student attitudes. 
Consequently, we searched AB1 Inform (a database of business periodicals) for studies 
evaluating students’ attitudes toward active learning, with a specific emphasis on selecting 
articles across sub-disciplines within accounting. When evaluating the individual survey 
items for inclusion in our survey, we intentionally selected ones that reflected the active 
learning experience and ones that could be generalized across different activities. Thus, our 
survey consists of a compilation of items selected from the articles discussed in the following 
section. 


Selection of Specific Survey Items 

We used Montano, Cardoso, and Joyce (2004) as a starting point since this article 
provides the most comprehensive list of survey items concerning students’ attitudes toward 
active learning within an accounting setting. Their article includes 40 items, assessing the 
following sub-categories: content learning, skill development, motivation, general 
assessment, and specific questions about the activities. Since our desire was to create scales 
with broad applicability, we removed items that were not generic in nature. In all, we 
selected 20 of the Montano et al. survey items. 

We then expanded our survey instrument by including items from four other 
accounting-related active learning articles: Chu and Libby (2010); Murphy (2005); Morse, 
Ruggieri, and Whelan-Berry (2010); and Scofield and Dye (2009). We selected an 
additional 21 survey items from these articles with two purposes: 1) to select items of a 
generic nature that address the active learning experience and 2) to provide an incremental 
contribution to the Montano et al. items. Thus, our survey included 41 items regarding 
students’ attitudes toward the active learning experience, which were rated on a seven-point 
scale, where l=strongly disagree and 7=strongly agree. 
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METHOD 

Sample Description 

The survey was administered at the conclusion of Managerial Accounting Principles 
courses where four hands-on, in-class active learning activities were implemented. The 
survey’s first page provided students with a brief description of the active learning activities 
to anchor their answers to the general active learning experience associated with these four 
activities. 

Students were in three classes at one private university and two classes at a different 
private university. We received 120 usable responses, which represented a 90% response 
rate for the students enrolled as of the semester’s end. To avoid introducing bias into 
students’ responses, the survey was administered by a colleague at each university. The 
students were informed through a pre-prepared script that their professor would not be 
given access to information about their participation or survey responses until after the 
semester ended. 

Analysis of the demographic information indicates that 42.5% of our respondents 
were female and were 20.6 years old on average. The students were mostly full-time 
(97.5%) with cumulative GPAs averaging 3.2. A major in business was reported by 95.8%, 
with 23.9% of the respondents indicating accounting or accounting combined with another 
business major. As expected, 56.3% of the students were sophomores with juniors 
constituting another 31.9%. 


Scale Development 

The purpose of factor analysis is to determine the fundamental dimensions that 
underlie a group of survey items (Hair, Black, Babin, Anderson, & Tatham, 2006). Thus, 
we undertook an exploratory factor analysis (EFA) with all 41 of the survey items assessing 
students’ attitudes toward the active learning experience. Per Tabachnick and Fi dell (2001), 
we used maximum likelihood as the factor extraction method (to increase the possibility of 
yielding the population correlation matrix) and promax as the rotation technique (an oblique 
method as the resulting factors are expected to be subscales of an overall student attitude, 
and thus, correlated). Also, we used minimum factor loadings of 0.50 to determine 
acceptable loadings, as Hair et al. (2006) indicate that level as “practically significant.” In 
determining the number of factors, we relied on the common eigenvalue minimum of 1.0. 
In addition, according to Hair et al.’s (2006) guidelines, our sample size is sufficient for 
identifying significant factors. 
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We commenced a procedure for interpreting the factor matrix. For this evaluation, 
we relied on five steps in the process described by Hair et al. (2006). First, we examined 
the factor loadings generated in the factor matrix. Not surprisingly, this EFA produced 
unclear results. During this initial evaluation, 14 of the 41 items either had significant cross¬ 
loadings or failed to load (given our minimum of 0.50). In other words, this initial EFA 
failed to produce a simple structure for the survey items (i.e., where an item has one 
significant loading on one factor) that resulted in distinct constructs. Second, we reviewed 
each item and identified the significant loadings of each, across the factors. Third, we 
examined the communalities for each of the 41 items. These values indicate the amount of 
variance for each item that is accounted for by the factor solution (Hair et ah, 2006). We 
identified four items that were not sufficiently explained through the factor analysis using 
the 0.50 guideline. 

The fourth step is to determine if the factor model should be re-specified. Thus, the 
goal is to make a decision regarding how problematic items are treated. For instance, Hair 
et al. (2006) list these problematic items as ones that (a) have no significant loadings, (b) 
have too low of a communality, or (c) have a significant cross-loading. One goal of factor 
analysis is data reduction that enables the replacement of the original data variables with a 
set of representative variables that facilitate subsequent statistical analysis. Thus, the 
objective of factor analysis is to minimize cross-loadings and “make each variable associate 
with only one factor’ ’ (H air et al., 2006). Therefore, we eliminated items that failed to 
significantly load on a factor, resulted in a low communality, or provided significant cross¬ 
loadings. At this stage, the remaining set included 26 items. To keep as many items in the 
analysis as possible, we re-estimated the EFA by re-considering each of the 15 eliminated 
items, as we wanted to determine if the elimination of an item might correct a cross-loading 
issue for another item. After completion of this process, the factor analysis produced a 
four-factor solution with 27 items that each significantly loaded on only one factor. Before 
proceeding to the fifth step, which involves labeling the factors, we first provide information 
regarding the empirical examination of the factors. 

ANALYSES 

After we finalized the factor solution, we followed the guidelines from Nunnally and 
Bernstein (1994) to examine the factors, which included undertaking confirmatory factor 
analyses (CFA), computing Cronbach’s alphas, and reviewing the range of responses. 
Once again, we used maximum likelihood extraction along with promax rotation. These 
factor loadings are provided in Table 1. A CFA for each of the four factors yields a single 
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factor. All loadings exceed the 0.50 guideline proposed by Tabachnick and Fidell (2001). 
Thus, evidence is provided for the uni-dimensionality of the scales. 


Table 1. Rotated factor matrix - confirmatory factor analysis final solution 


Cronkack’s 

alpka 

Impact on 

Studying for Current Class (variance explained of 74.8%) 

0.951 

Item 

Factor 


No 

Loading 



1 

I used what 1 learned from the activities to study for the 




exams. a 

.866 


The hands-on activities... 



2 

helped me to clarify the most difficult contents of the 
subject by making them easier to understand. 

.821 


3 

made me study better. b 

.849 


4 

motivated me to work harder in the class. b 

.765 


5 

helped me in preparing for examinations. c 

.901 


6 

changed my attitude in the way I approach my studying. b 

.787 


7 

provided additional help (beyond merely doing the 
homework) in terms of learning managerial accounting. a 

.819 


8 

helped me better prepare for the exams. c 

.927 


Perception of Authentic Value (variance explained of 78.2%) 


0.953 

1 

I think that the time devoted to the activities was 
worthwhile. b 

.885 


2 

I believe including the active learning activities in this 
course was useful. b 

.836 


3 

I wish these types of activities were used in all my classes. d 

.842 


4 

I would like to see more active learning activities in my 
future classes. c 

.892 


5 

All things considered, 1 believe that the active learning 
activities were worthwhile. b 

.928 


6 

I like this type of hands-on activity more than the 
traditional class lecture. c 

.793 


7 

I would like it if this type of hands-on activity was used in 
other courses. c 

.864 
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Att 

itude toward Current Class (variance explained of 73.5%) 


0.927 

The hands-on activities... 



1 

helped me feel positive towards the accounting class. e 

.868 


2 

made it more comfortable for me to participate in class 
discussion. d 

.802 


3 

helped me to understand, widening and relating my ideas. b 

.791 


4 

improved my opinion on the contents of the class. b 

.833 


5 

helped me feel positive towards accounting. e 

.842 


6 

made me feel more actively involved in the learning process 
for managerial accounting. a 

.817 


Interest in Current Class (variance explained of 68.2%) 


0.903 

1 

The active participation during the activities made the class 
more interesting. b 

.835 


2 

I found that the activities made the topic of managerial 
accounting more interesting. a 

.856 


3 

In general, 1 think these activities reveal the teacher’s 
concern for quality teaching. b 

.601 


4 

The hands-on activities have been interesting. b 

.787 


5 

The hands-on activities allow sharing of ideas, responses 
and points of view with my peers and teachers. 

.734 


6 

The hands-on activities made the class more interesting. a 

.883 



Only the factor loadings exceeding 0.50 are included in the table. 

Extraction Method: Maximum Likelihood. Rotation Method: Promax with Kaiser Normalization. 


Items were modified from: 1 Murphy (2005); b Montano et ah (2004); c Chu and Libby (2010); d Morse 
et al. (2010); e Scofield and Dye (2009). 

The correlation matrix in Table 2 reports in the diagonal the Cronbach’s alpha for 
each scale. All of the values exceed the generally accepted cutoff of 0.70 advocated by 
Nunnally and Bernstein (1994), as well as the less stringent cutoff of 0.60 for exploratory 
research such as what was done in this paper (Hair et ah, 2006). Therefore, each factor 
demonstrates reliability. Also in Table 2 are the correlation coefficients between the pairs 
of constructs. A comparison of the Cronbach’s alpha and each of the correlations shows 
that the correlation within each construct is higher than the correlation across constructs. 
Thus, evidence of discriminant validity is obtained. 
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Table 2. Reliability and correlation matrix: The Cronbacb’s alpha values are reported in 


the diagonal, 

while the other values 

represent the 

correlation coefficients. 

Studying 

Studying 

0.951 

Authentic 

Attitude Interest 

Authentic 

0.791** 

0.953 


Attitude 

0.720** 

0.707** 

0.927 

Interest 

0.642** 

0.737** 

0.729** 0.903 


** significant at p < 0.01 (2-tailed) 

Where: Studying=lmpact on Studying for Current Class; 

Authentic=Perception of Authentic Value; 

Attitude=Attitude toward Current Class; Interest = Interest in Current Class 

Most items use the full range of potential responses. A review of the kurtosis and 
skewness showed that all the variables demonstrate accepted levels according to general 
guidelines noted by Kline (2005) of < 3 for skewness and < 10 for kurtosis. 

Interpretation of Scales 

As the final step in the factor interpretation, we asked other accounting faculty to 
review the factor groupings to provide independent labels describing each one. Based on 
this feedback and our own review, we labeled the four factors: Impact on Studying for 
Current Class, Perception of Authentic Value, Attitude toward Current Class, and Interest 
in Current Class. 

The first factor, Impact on Studying for Current Class, consists of eight items that 
capture whether the active learning activities aided them when studying. Scale items include 
“The hands-on activities helped me better prepare for the exams.” The second factor, 
Perception of Authentic Value, measures students’ opinions about the extent to which the 
active learning activities were worth the time invested and whether they should be used in 
future classes. The seven items in this factor include, “I think the time devoted to these 
activities was worthwhile.” 

The third factor, Attitude toward Current Class, includes six items that assess the 
impact that the activities have on the students’ overall opinion toward the current class. 
The items evaluate whether, for example, “The hands-on activities improved my opinion on 
the contents of the class.” The fourth factor, Interest in Current Class, captured students’ 
attitudes regarding whether the active learning activities made the current class more 
interesting. The six items in this scale include “The active participation during the activities 
made the class more interesting.” 
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The empirical data collected result in these four distinct factors that now can be used 
to measure students’ attitudes and perceptions regarding active learning. In the next 
section, we supplement this empirical data with a qualitative review of student feedback. 

EXPLANATORY MIXED METHODOLOGY 

In explanatory mixed method design, triangulation consists of first collecting 
quantitative data and then collecting qualitative data to “help explain or elaborate on 
quantitative results” (Creswell, 2008, p. 560). In this study, factor analysis refined and 
reduced items on a survey instrument to form scales. These scales focused on the students’ 
perception of value added from the incorporation of active, authentic learning. Using 
qualitative inquiry, we followed this factor analysis with the thematic evaluation of open- 
ended questions designed to evaluate the course’s strong and weak points; questions whose 
analysis served as a secondary source to “increase scope, depth and consistency” (Flick, 
2002, p. 227) of the scales and provide an enriched student voice to the value of active, 
authentic learning experiences as quantified in the scales (Creswell & Plano Clark, 2007). 

We based the qualitative analysis on data gathered from the end-of-course student 
evaluation form for Researcher A. The university distributes this student evaluation of 
teaching (SET) instrument throughout the campus to collect student perceptions. 
Consequently, the SET did not prompt students to recall the active learning activities. Data 
from the SET were collected approximately one month after the last in-class activity was 
completed. We analyzed students’ responses to two open-ended questions before and after 
the introduction of active, authentic learning activities. These questions reveal students’ 
attitudes toward the overall course by asking them to comment on the 1) strong points of 
the course, and 2) weak points of the course. 

SET respondents consisted of 101 students, 38 before activities (BA) and 63 after 
activities (AA), who were enrolled in Researcher A’s course. We eliminated 79 of the 202 
potential student comments as the student did not provide a comment (n = 75) or the 
comment did not address the question asked (n = 4). Thus, we rel ied on 123 comments 
(44 BA, 79 AA). 

Multiple, intensive readings combined with constant comparison of survey responses 
by the researchers established the parameters for analytical groupings. inter-rater 
agreement was negotiated on the summative and salient attributes of each theme, subtheme, 
and category. 
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QUALITATIVE ANALYSIS 

Three themes on student attitudes toward strengths and weakness in the 
introductory managerial accounting course emerged: 1) Class Experience, 2) Relevance, and 
3) Course Materials. Each theme divided into four subthemes reflecting a two-by-two 
matrix of 1) columns titled Self-Directed versus Teacher-Directed Learning and 2) rows 
titled Recognition versus Lack of Recognition of the Value and Application of Knowledge. 
Comments classified as self-directed learning often were written in first person or related 
to the student’s personal experience, while teacher-directed comments emphasized 
decisions made by the instructor. 

As seen in Table 3, two of the themes, Class Experience and Relevance, further 
subdivided into several categories as discussed below. Table 3 presents the relative 
frequency of comments for each theme, subtheme, and category stated as a percentage of 
the total BA comments and AA comments, respectively. 

The Class Experience Theme and the Relevance Theme include many comments that 
are similar to and congruent with the four factors presented in Table 1. The Relevance 
Theme also addresses the authenticity of the learning environment. Combined, these two 
themes constitute over 85% of both the BA and AA student comments; consequently, they 
are the focus of the following discussion. 

Class Experience Tkeme 

The Class Experience Theme relates to students’ comments regarding different 
aspects of their experience in the introductory managerial accounting class. This theme 
comprised the majority of student comments both before and after the activities were 
introduced (BA = 55%, AA = 60%). Overall, students made more positive comments after 
the activities were introduced (BA = 25%, AA = 30%). Within each subtheme, four 
categories emerged: Testing, Presentation of Classroom Content, Difficulty, and 
Value/Interest/Attitude (see Table 3). 

The category Value/Interest/Attitude dominated the BA comments (30%), with 
comments like “Good stuff to know” and “Important and seems practical and useful.” In 
contrast, the most frequent AA comments relate to the category Presentation of Classroom 
Content (22%) with statements such as “The activities we did in class were helpful and 
explained the work very clearly.” The noticeable increase in positive comments for this 
category (BA = 5%, AA = 17%) highlights the lasting impression of the active learning 
activities. 
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The Class Experience Theme includes all comments that specifically mention the 
active, authentic learning activities. Hence, the comments provided by the AA students 
give voice to the four-factor solution presented in Table 1. Table 4 contains selected student 
comments that convey similar sentiments as each factor and allow for a richer factor 
interpretation. The selected student comments appeared on the end-of-course evaluation 
form after implementation of the authentic active learning activities and are in response to 
the open-ended prompt: “Strong Points- I he Course.” The comments convey similar 
sentiments as the four-factor solution presented in Table 1. 

Table 3. Qualitative Summary of Themes in Student Comments: relative frequency of 
student comments on the end-of-semester course evaluation forms before and after the 
introduction of the authentic active learning activities 


Panel A: Class Experience Theme 



Before Activities 

After Activities 


(n=24/44=55% 

overall) 

(n=48/79= 

=61% overall) 



Teacher- 

Self- 

Teacher- 


Self-Directed 

Directed 

Directed 

Directed 

SUBTHEMES 

Learning 

Learning 

Learning 

Learning 

Recognition of the Value 
and Application of 
Knowledge 

16% 

9% 

15% 

15% 

Testing 

... 

4% 

1% 

1% 

Presentation of 

Classroom Content 


5% 

4% 

13% 

Difficulty 

... 

... 

3% 

... 

Value/Interest/Attitude 

16% 

... 

7% 

1% 

Lack of Recognition of the 
Value and Application of 
Knowledge 

9% 

21% 

5% 

25% 

Testing 

2% 

2% 

1% 

8% 

Presentation of 

Classroom Content 


3% 


5% 

Difficulty 

2% 

7% 

1% 

7% 

Value/Interest/Attitude 

5% 

9% 

3% 

5% 
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Panel B: Relevance Theme 



Before Activities 

After Activities 


(n=16/44= 

=36% overall) 

(n=21/79= 

=26% overall) 


Self- 

Teacher- 

Self- 

Teacher- 


Directed 

Directed 

Directed 

Directed 

SUBTHEMES 

Learning 

Learning 

Learning 

Learning 

Recognition of the Value and 
Application of Knowledge 

18% 

18% 

15% 

8% 

Work-Related Authenticity 

13% 

... 

10% 

3% 

Course-Specific Authenticity 

5% 

18% 

5% 

5% 

Lack of Recognition of the 
Value and Application of 
Knowledge 




4% 

Work-Related Authenticity 

... 

... 

... 

1% 

Course-Specific Authenticity 

... 

... 

... 

3% 


Panel C: Course Materials Theme 



Before Activities 

After Activities 


'St' 

'Sf 

II 

a 

=9% overall) 

(n=10/79= 

=13% overall) 


Self- 

Teacher- 

Self- 

Teacher- 


Directed 

Directed 

Directed 

Directed 

SUBTHEMES 

Learning 

Learning 

Learning 

Learning 

Recognition of the Value and 





Application of Knowledge 

2% 

... 

5% 

1% 

Lack of Recognition of the 





Value and Application of 





Knowledge 

... 

7% 

3% 

4% 

The frequencies are reported as a percentage of the total comments made either before th 
(n=44) or after the activities (n-79), respectively. 

e activities 
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Table 4. Selected student comments related to the Four-Factor Solution 

Factor 1: Impact on Studying for Current Class 

In class activities. Explains process thoroughly. 

Material that we go over in class is relevant to what is covered on the exam. 

Liked the different exercises we performed so that we could better understand the 
material. 

Factor 2: Perception of Authentic Value 

Material is very important for real life work. 

A lot of useful information that will be used in everyday business life. 

In class activities, examples done in class. 

Factor 3: Attitude toward Current Class 
I have learned a lot about managerial accounting which will help me in the future. 
Interactive. 

Very informative, equips us with relevant skills and knowledge. 

Factor 4: Interest in Current Class 

I liked that we did the in-class exercises/activities because it helped mix up the class. 
She did the hand on exercises that really help as well as makes the class fun. 

It was very interesting material. 


Relevance Theme 

The Relevance Theme includes student comments about how the introductory 
managerial accounting course prepared students for real life work and provided appropriate 
content. Within each subtheme, two categories emerged: Work-Related Authenticity and 
Course-Specific Authenticity. Both the BA and AA students commented with similar 
frequency on Work-Related Authenticity (13% and 14%, respectively). For example, one 
student wrote: “This course is great in that it will help in the future with making important 
decisions for a firm!” 

Comments made by students under the Relevance Theme predominantly speak to 
their valuing of authentic learning in the classroom. Student comments in the Relevance 
Theme also correspond to the four-factor solution presented in Table 1. 

Ultimately, this qualitative analysis documents students’ perceptions and attitudes 
toward the course and its content and served as a secondary explanation to the quantitative 
results. Themes and student comments that supported these themes resulting from factor 
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analysis expanded quantitative results and gave voice to the underlying construct within the 
factors revealed (Creswell & Plano Clark, 2007). 

ADDITIONAL ANALYSES 

Our next step is to empirically examine the difference between the reliance on a 
single question versus a multiple-item scale. We apply two approaches to studying this 
issue: 1) determine the reliability of the single-item measure and 2) testing a structural 
equation model of expected outcomes for the measures. 

Loo (2002) computed the reliability of single-item scales in comparison to estimated 
minimum reliability benchmarks. His calculations applied Spearman’s “classical formula 
for the correction for attenuation” as follows (2002, p. 68): 

7 xy yj^xx yj r yy 

Where: r xy = the correlation between variables 
r xx = the reliability of the single item x 
r yy = the reliability of the multi-item scale y 

Wanous, Reichers, & Hudy (1997) state a reasonable minimum estimated reliability 
of a single item of approximately 0.70. We calculated the correlation between the scale 
and the item that resulted in the highest load on that factor. By substituting this number, 
along with the scale’s Cronbach’s alpha, we computed the following estimates of reliability 
for the single item: Impact on Studying Scale, Item 8 - 0.947; Perception of Authentic 
Value, Item 5 - 0.948; Attitude toward Current Class, Item 1 - 0.900; and Interest in 
Current Course, Item 6 - 0.919. Therefore, this analysis suggests that use of a single item 
to measure these constructs may be appropriate. 

To apply a more rigorous analysis of the issue, we constructed a structural equation 
model testing our scales as antecedents to students’ approaches to learning. This model is 
based on the research by Lizzio et al. (2002) who depict the learning environment as an 
antecedent to the same approaches to learning scales. We then estimated the model using 
SPSS AMOS version 20. The model applying the single item fails to generate a model 
with acceptable fit indices. In contrast, the model that includes the multiple item scales 
provided acceptable fit indices. 

Our goal with this analysis is not to test hypotheses. Therefore, we will not delve 
into an analysis of each relationship. Our purpose is to determine if single items are 
acceptable in estimating models relating our scales to expected outcomes. Examining the 
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results from this perspective indicates that the single item did not result in a useable model. 
The Hoelter’s Critical N supports our contention that sample size is not a restriction in 
this case, as our sample exceeds the minimum level specified by that measure. 

CONCLUSION 

Fuchs and Diamantopoulos (2009) demonstrate that the variation in attitude and 
perceptions necessitates multi-item scales to fully capture the constructs of interest. This 
occurrence is especially prevalent in studying the efficacy of active learning. These scales 
provide researchers with the building blocks to investigate and model the role that student 
attitudes (i.e., affective domain) have on specific learning outcomes (i.e., cognitive domain). 

To improve the scales’ generalizability, we relied on existing survey items that have 
already been used to measure students’ attitudes regarding active learning experiences. We 
specifically selected items from articles across accounting sub-disciplines with different 
activity characteristics. We undertook explanatory mixed methodology using the end-of- 
course commentaries to conduct a follow-up explanation model to expand quantitative 
results (Creswell & Plano Clark, 2007). In both the scales and the deconstruction of 
dialogue, students related to issues of class experience, relevance of the course, and course 
materials. 

We recognize that the development of these scales was done within the context of a 
single course with a limited set of activities, in addition, we worded the survey items to 
focus students’ responses on our learning activities. Thus, we anticipate that some wording 
may require modification for future studies. However, minor variations that retain the 
primary stem of the items would not be expected to change the applicability of the scales 
across settings. Nonetheless, these issues related to generalizability highlight the need for 
future researchers to assess the stability of these scales across different samples and 
activities. Ultimately, we envision these scales as bei ng a necessary first step toward the 
ability of future researchers to efficiently and effectively test a more comprehensive model 
of the benefits of active learning. 
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